Goto

Collaborating Authors

 singapore management university


Efficient Test-Time Retrieval Augmented Generation

Yin, Hailong, Zhu, Bin, Chen, Jingjing, Ngo, Chong-Wah

arXiv.org Artificial Intelligence

Although Large Language Models (LLMs) demonstrate significant capabilities, their reliance on parametric knowledge often leads to inaccuracies. Retrieval Augmented Generation (RAG) mitigates this by incorporating external knowledge, but these methods may introduce irrelevant retrieved documents, leading to inaccurate responses. While the integration methods filter out incorrect answers from multiple responses, but lack external knowledge like RAG methods, and their high costs require balancing overhead with performance gains. To address these issues, we propose an Efficient Test-Time Retrieval-Augmented Generation Framework named ET2RAG to improve the performance of LLMs while maintaining efficiency. Specifically, ET2RAG is a training-free method, that first retrieves the most relevant documents and augments the LLMs to efficiently generate diverse candidate responses by managing response length. Then we compute the similarity of candidate responses and employ a majority voting mechanism to select the most suitable response as the final output. In particular, we discover that partial generation is sufficient to capture the key information necessary for consensus calculation, allowing us to effectively perform majority voting without the need for fully generated responses. Thus, we can reach a balance between computational cost and performance by managing the response length for the number of retrieved documents for majority voting. Experimental results demonstrate that ET2RAG significantly enhances performance across three tasks, including open-domain question answering, recipe generation and image captioning.


Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol

Ma, Wei, Yang, Yixiao, Hu, Qiang, Ying, Shi, Jin, Zhi, Du, Bo, Xing, Zhenchang, Li, Tianlin, Shi, Junjie, Liu, Yang, Jiang, Linxiao

arXiv.org Artificial Intelligence

Applications of Large Language Models~(LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions. Their inherent non-determinism, dynamism, and context dependence pose fundamental challenges for quality assurance. This paper decomposes LLM applications into a three-layer architecture: \textbf{\textit{System Shell Layer}}, \textbf{\textit{Prompt Orchestration Layer}}, and \textbf{\textit{LLM Inference Core}}. We then assess the applicability of traditional software testing methods in each layer: directly applicable at the shell layer, requiring semantic reinterpretation at the orchestration layer, and necessitating paradigm shifts at the inference core. A comparative analysis of Testing AI methods from the software engineering community and safety analysis techniques from the AI community reveals structural disconnects in testing unit abstraction, evaluation metrics, and lifecycle management. We identify four fundamental differences that underlie 6 core challenges. To address these, we propose four types of collaborative strategies (\emph{Retain}, \emph{Translate}, \emph{Integrate}, and \emph{Runtime}) and explore a closed-loop, trustworthy quality assurance framework that combines pre-deployment validation with runtime monitoring. Based on these strategies, we offer practical guidance and a protocol proposal to support the standardization and tooling of LLM application testing. We propose a protocol \textbf{\textit{Agent Interaction Communication Language}} (AICL) that is used to communicate between AI agents. AICL has the test-oriented features and is easily integrated in the current agent framework.


Google backs six artificial intelligence-based research projects – Details

#artificialintelligence

Artificial Intelligence (AI) is opening up the next phase of technological advances. Riding the AI wave, Google has started six AI-based research projects in India. These projects would focus on addressing social, humanitarian and environmental challenges in sectors such as healthcare, education, disaster prevention and conversation. Google Research India, based in Bengaluru, will provide funding and computational resources besides supporting the efforts with expertise in computer vision, natural language processing, and other deep learning techniques, says Manish Gupta, director of Google Research Team in India. The research team will focus on two pillars: First, advancing fundamental computer science and AI research by building a strong team and partnering with the research community across the country and secondly, applying this research to tackle big problems in fields such as healthcare, agriculture and education while also using it to make apps and services more helpful.


The 2018 Survey: AI and the Future of Humans

#artificialintelligence

"Please think forward to the year 2030. Analysts expect that people will become even more dependent on networked artificial intelligence (AI) in complex digital systems. Some say we will continue on the historic arc of augmenting our lives with mostly positive results as we widely implement these networked tools. Some say our increasing dependence on these AI and related systems is likely to lead to widespread difficulties. Our question: By 2030, do you think it is most likely that advancing AI and related technology systems will enhance human capacities and empower them? That is, most of the time, will most people be better off than they are today? Or is it most likely that advancing AI and related technology systems will lessen human autonomy and agency to such an extent that most people will not be better off than the way things are today? Please explain why you chose the answer you did and sketch out a vision of how the human-machine/AI collaboration will function in 2030.


Professionals urged to upskill as AI reshapes finance sector: CPA

#artificialintelligence

CHIEF financial officers (CFOs) and their finance teams need to understand how to anticipate and respond to artificial intelligence (AI), as it will be a key frontier technology to grow Singapore's economy in the years ahead, CPA Australia said on Thursday. The accounting professional body also urged professionals in the sector to upskill, particularly in terms of data mining, extraction and faster interpretation of big data. To help them navigate this digital journey, CPA Australia has published a resource titled Charting the Future of Accountancy with AI, in collaboration with Singapore Management University's School of Accountancy. The practical guide looks at how AI will reshape the accounting and finance sector in the coming years, and what the profession can do to continue to operate alongside the evolving technology and their changing roles. It draws on insights from professional services firms Accenture, Deloitte, EY, KPMG and PwC, as well as the Singapore Management University.


Artificial Intelligence Research in Singapore: Assisting the Development of a Smart Nation

Varakantham, Pradeep (Singapore Management University) | An, Bo (Nanyang Technological University) | Low, Bryan (National University of Singapore) | Zhang, Jie (Nanyang Technological University)

AI Magazine

Artificial intelligence (AI) research in Singapore is focused on accelerating the country's development into a smart nation. Specifically, AI has been employed extensively in either augmenting the intelligence of humans or in developing automated methods and systems to improve quality of life in Singapore. In this column we summarize Singapore's Our focus in this column is primarily limited to the efforts of Singapore to become a smart nation. The key areas of AI research summarized here include mobility, security, manufacturing, and health care. In addition, there are also translational domain has taken a number of interesting directions.